METHOD AND APPARATUS FOR GENERATING STRUCTURAL 
DATA FROM LASER REFLECTANCE IMAGES 

FIELD OF THE INVENTION 

The present invention relates generally to a method and 
apparatus used to document the spatial relationships and geometry 
of existing buildings and structures. Specifically, the present 
invention relates to a method and apparatus comprising a 3D camera 
(scanning laser range finder) which can produce high resolution 
reflectance images, a magnetic storage device which can store 
scanned measurement data, a set of algorithms which are used to 
interpret the measurement data, and a software interface which 
allows a human to interact and query the measurement information in 
useful ways. 

BACKGROUND OF THE INVENTION 

Revamp work is a significant portion of factory and plant 
engineering activities. Factories and plants are constantly 
updating processes and revamping production practices to meet new 
product demands, EPA regulations, and to replace worn out 
equipment. As a result, architects and engineering firms have to 
design equipment that can interface with existing plant conditions. 
For example, a new process may require interfacing with hundreds or 
even thousands of existing pipe runs. In order to design the 
equipment - primarily piping - measurements must be made of the 
existing factory or plant. 

Existing design drawings are not good enough for this 
task for three primary reasons: 

1. They cannot be found or do not exist due to poor 
record keeping 

2. The drawings are pre-CAD and all measurements must 
be manually interpreted 



3. The plant construction does not match the design to 
the tolerances required for the refit work. 

Measuring the existing conditions of an operating plant is 
considered to be: 

1. Dangerous. The plant cannot be shut down for 
measurement work so dangerous conditions exist 
ranging from moving equipment to hot equipment to 
exposure to caustic substances. In addition, much 
of what must be measured is high in the air or 
otherwise inaccessible . 

2. Expensive. Field measurement requires sending a 
crew of workers to the site for an extended period 
of time. This is expensive due to man-hour costs 
on site, lost productivity, travel and lodging 
expenses, and costs of labor associated with plant 
personnel required for escort functions. 

3. Slow. Current measurement techniques are measured 
in units of weeks and months. This can seriously 
impact production schedules. Each day a plant is 
not operating represents lost production capacity 
that can never be recovered, and lost profits that 
can never be realized. 

4. Inaccurate. Mistakes in measurement, or the 
inability to measure parts of the plant, account 
for serious costs in construction change orders and 
rework . 

Existing techniques for performing as -built measurement 
include manual methods using tape measures and plumb bobs, 
traditional survey techniques, photogrammetry (which is metrology 
using stereo photograph pairs - often used in aerial surveying and 



more recently used for process plant measurement) , and most 
recently laser measurement using scanned laser radar systems. 

Manual techniques are extremely slow, cumbersome, and 
dangerous due to needs for hands-on measurement. They are also 
subject to error and cannot reliably deliver the accuracy required 
by the engineering profession (+/- 1/4 inch) . Manual techniques 
for our purposes comprise those techniques which employ traditional 
measurement tools including tape measures, plumb bobs, rules, and 
sketch pads. 

Traditional survey techniques are useful for ground-based 
structures but are slow and require a very specialized work process 
to achieve good results. In addition, the number of data points 
that can be affordably measured is very small due to the labor- 
intensive nature of the techniques. Errors can be hard to detect 
but can drastically influence the outcome due to the small amounts 
of data gathered. For the purpose of this document, traditional 
survey techniques comprise the set of techniques that utilize 
theodolites, laser total stations, retro-reflective targets, and 
other support equipment that is commonly used by industrial and 
land surveyors. 

Photogrammetry can be more accurate than manual methods 
and more comprehensive than traditional survey techniques but is 
very slow and cumbersome. This technique utilizes registered pairs 
of photographs to triangulate distances from the cameras to points 
on the surfaces of the plant structures. Photogrammetry relies on 
an accurate traditional ground survey in order to develop scale, 
and takes weeks to months for post data collection processing to 
produce useable results. 



Laser radar (also called lidar) is the newest measurement 
technique to be applied to plant measurement. With this technique 
a laser is scanned across a volume of space and distances to the 
surfaces of objects are computed based on the properties of the 
returning laser beam. Typically the set of points generated by the 
lidar device is processed using mathematical algorithms into models 
that can be utilized by traditional CAD software. 

The present invention is comprised of a lidar device but 
with unique properties that enable useful structural data to be 
generated faster and with a completely different paradigm than 
other lidar systems. 

SUMMARY OF THE INVENTION 

The present invention pertains to an apparatus for 
generating structural data of a structure. The apparatus comprises 
a scanning laser range finder that produces reflectance data of the 
structure. The apparatus comprises a memory which stores the 
reflectance data. The apparatus comprises means for determining 
desired spatial relationships of the structure from the reflectance 
data . 

The present invention pertains to a method for generating 
structural data of a structure. The method comprises the steps of 
producing reflectance data of the structure with a scanning laser 
range finder. There is the step of storing the reflectance data in 
a memory. There is the step of determining desired spatial 
relationships of the structure from the reflectance data. 

The present invention is a system for generating 
structural data from laser reflectance images. By structural data 
it is meant spatial measurements of built environments including 



but not limited to: distance between two discrete points, diameter 
of pipes, size of structural steel, distance between pipe centers, 
direction and orientation of pipe runs, distance to center of pipe 
flanges, distance between walls or between floors and ceilings, and 
the location and orientation of planar surfaces. 

A scanning laser camera collects data and generates both 
a point cloud and a high-resolution reflectance image 
simultaneously. The reflectance image is loaded into a software 
interface that allows a human operator to query the data to 
determine the types of measurement data as listed above. 
Manipulation by the human operator of the 3D point data is not 
required to generate the information which leads to faster 
operation, lower demands on computing resources, and less cost to 
generate required information. 

The present invention is unique in that it works 
exclusively with a 2D representation of a 3D database. All of the 
measurement functions currently requiring teams of field personnel 
to accomplish can be done in a fraction of the time in the comfort 
of a home office. The data will be more complete and more accurate 
and can be manipulated easily with workers unskilled in 3D data 
manipulation. The speed of useful data extraction will be orders 
of magnitude faster than any other technique both because of the 
high-speed 3D camera used to collect the data and because there is 
no need to generate a 3D solid based CAD model from which to 
generate measurements - as is typically done with 3D point data. 
This last point will save weeks of work on a typical job and will 
make viable a whole new class of applications which heretofore 
could not afford to employ advanced measurement techniques. 
Overall the benefits of the present invention will result in lower 
engineering costs, lower construction costs, and a US industrial 
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base that will have a more accurate, better-documented 
infrastructure . 

BRIEF DESCRIPTION OF THE DRAWINGS 

In the accompanying drawings, the preferred embodiment of 
5 the invention and preferred methods of practicing the invention are 
illustrated in which: 

Figure 1 is a block diagram of the system in accordance 
with an embodiment of the invention. 

Figure 2 is a block diagram that illustrates one 
10 embodiment of a three-dimensional camera. 

Figure 3 is a block diagram illustrating the RDM module 
in its broadest aspect. 

Figure 4 is a screen shot showing the first embodiment of 
the RDM module. 

15 Figure 5 is a screen shot showing a detail of the image 

window with a reflectance image loaded. 

DETAILED DESCRIPTION 

Referring now to the drawings wherein like reference 
numerals refer to similar or identical parts throughout the several 
2 0 views, and more specifically to figure 1 thereof, there is shown an 
apparatus for generating structural data of a structure. The 
apparatus 12 comprises a scanning laser range finder 22 0 that 
produces reflectance data of the structure. The apparatus 12 
comprises a memory 60 which stores the reflectance data. The 
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apparatus 12 comprises means 14 for determining desired spatial 
relationships of the structure from the reflectance data. 

Preferably, the determining means 14 includes interface 
means 16, such as a key board, for allowing desired spatial 
5 relationships to be chosen. The data preferably includes a 
reflectance image 270 and a point cloud 3 0 of the structure that 
are produced simultaneously by the laser range finder 220. 
Preferably, the reflectance image is provided to the interface 
means 16 which allows a user to work with the reflectance image 
u 10 that is a 2D representation of the point cloud 30 which is a 3D 
b representation of the structure. The data preferably includes 
hi distance measurements 40 and reflectance measurements 50 of the 
*C structure that have a direct correspondence. The reflectance image 
H can be formed from reflectance measurements and range data. 

s. 15 The determining means 14 preferably determines a location 

of a discrete point in absolute coordinates of the structure; or 
FU distances between two discrete locations of the structure; or 
g perpendicular and straight line distances between two edges in the 
fU structure; or diameters, centerlines and orientations of pipes in 
2 0 the structure; or perpendicular distances between pipe centerlines 
of the structure; or dimensions of structural elements; or nominal 
or standard sizes of components of the structure; or dimensions of 
flanges, nozzles and other pipe connections of the structure; 
distances between two planes of the structure; or 2D area 
25 measurements of the structure; or 3D volume measurements of the 
structure; measurements of multiple reflectance images of the 
structure; or location and orientation of planar surfaces of the 
structure . 

The present invention pertains to a method for generating 
30 structural data of a structure. The method comprises the steps of 
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producing reflectance data of the structure with a scanning laser 
range finder 220. There is the step of storing the reflectance 
data in a memory 60. There is the step of determining desired 
spatial relationships of the structure from the reflectance data. 

Preferably, the determining step includes the step of 
choosing desired spatial relationships. The producing step 
preferably includes the step of producing a reflectance image 2 70 
and a point cloud 3 0 of the structure simultaneously by the laser 
range finder 220. Preferably, the choosing step includes the step 
of providing the reflectance image to an interface means which 
allows a user to work with the reflectance image that is a 2D 
representation of the point cloud 3 0 which is a 3D representation 
4? of the structure. The data preferably includes distance 
r\ measurements and reflectance measurements of the structure that 
hJL5 have a direct correspondence. Preferably, the producing the 
reflectance image step includes the step of forming the reflectance 
image from reflectance measurements and range data. 



gas. 

in 



I In the operation of the invention, figure 1 is a block 

J 

J diagram that illustrates the invention in its broadest aspect. 

20 Referring to figure 1, a 3D camera 20 senses the position in three- 
dimensional space to points on an object (s) 10 generating a point 
cloud 3 0 consisting of both distance measurements 40 and 
reflectance measurements 50 of the object (s) 10. 

The point cloud 30 is stored on a magnetic media 60 for 
25 use by the determining means 14, such as a Reflectance Display & 
Measurement (RDM) module 70. 



The reflectance measurements 50 are processed into a 
reflectance image 270. The reflectance image 270 is stored on a 
magnetic media 60 for used by the RDM module 70. 
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The RDM module 70 loads the point cloud 3 0 from the 
magnetic media 60 into a Graphical Interface 80. Interacting with 
the reflectance image 2 70 generates requests for structural 
information 100. A direct correspondence between the reflectance 
5 measurements 50 and the distance measurements 40 allows the 
measurement algorithm module 90 of the RDM module 70 to generate 
geometric relationships using the entire point cloud 30. 



The calculated structural information 10 0 (distances 
between points, pipe diameters, center distances, etc.) is 
ydO displayed in the Graphical Interface 80 but alternately can be 
O stored on a magnetic media 60 or used as input to conventional CAD 
gTJj tools 110. 

fi Figure 2 is a block diagram that illustrates the 

lil governing principles of one embodiment of the 3D camera. Also 
LJ.5 refer to U.S. Patent No. 6,034,803, incorporated by reference 
5 herein. Referring to figure 2, a laser diode 200 is used to 
W produce a continuous beam of energy 210. The energy is steered 
n through a continuous 360-degree azimuth angle and an 80-degree 
Rj vertical elevation angle to completely cover the object (s) 10 using 
2 0 a scanning mechanism 22 0 described in U.S. Patent No. 6,034,803. 
The energy is reflected off of the object (s) 10 and is returned 
through the scanning mechanism 22 0 where it is sampled by a photo- 
diode 230. A Range Measurement Module (RMM) 240 converts the 
sampled energy into a distance measurement 4 0 and a reflectance 
25 measurement 50. The distance measurement 40 is expressed in 
spherical coordinates and comprises the azimuth angle, zenith 
angle, and distance to a point on an object from the zero point of 
the camera. The zero point of the camera is located at the 
geometric center of the scanning mirror surface. The reflectance 
30 measurement 50 represents the amount of energy contained in the 
reflected energy used to generate the distance measurement 40. 



The 3D camera used in this embodiment of the invention is 
manufactured by Quantapoint, Inc. and operates at a high speed 
producing 125,000 discrete data points every second. This high 
rate of data capture allows measurements to be spatially very dense 
while still producing enough volume coverage per unit time to be a 
commercially viable measurement device. Because of the high 
spatial density, the resulting reflectance measurements 50 can be 
formatted using industry standard file formats (such as TIFF or 
JPEG) into a reflectance image 270 that can be viewed similar to a 
traditional photograph. The reflectance image 270 can be viewed as 
a gray- scale image where each pixel element of the image 
corresponds uniquely to a distance measurement 40. In this 
embodiment of the invention, each reflectance measurement 50 
captured by the 3D camera is 16 bits wide. When displayed to an 
operator, the reflectance image 270 data is compressed to 8 bits 
using a JPEG format to reduce computational load. 

The raw intensity data is seldom suitable for detailed 
work because it is not optimized for operator viewing. Since the 
invention produces images using an internal source of illumination, 
the raw images show great contrast between near and far regions. 
This contrast unnecessarily confuses the image and makes it 
difficult to work with, especially in comparison with photographs 
taken with external illumination. 

Using the following mathematical correction model for the 
generation of viewable images: 

R xy = V(I(i xy ) * B(r xy )) , (A) 
Where 

r xy = Physical range observed from pixel (x, y) , 
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i xy = Physical intensity observed from pixel (x, y) , 
B(r) = "Boost" correction factor, 
I (i) = Intensity correction factor, 
R = Final reflectance value, 
5 V = Overall transfer function. 

The boost function B compensates for the falloff of 
intensity as a function of range. Theoretical simulations indicate 
that B is determined by the geometry of the lensing system within 
the laser acquisition module as well as the (l/r 2 ) drop due to 
y:10 viewing distance. It is simplest to determine B by interpolating 
Q direct measurements at periodic intervals throughout the operating 
fi range of the device according to the following procedure: 

1) Place a target of consistent reflectance at a known 
distance from the 3D camera. 

2) Determine the raw intensity measured at this 
distance . 

3) Repeat steps 1 & 2 every few meters over the useful 
range of the device, generating a set of ordered 
pairs, { (r n/ i n ) }. 

4) Approximate B(r n ) = (i 0/ i n ) • 

5) Determine B for all other ranges via interpolation 
or curve fit. 

The intensity correction factor is determined similarly, 
using multiple targets of varying (but known) reflectance at a 
25 fixed distance. The overall transfer function V is inserted to 
provide a mechanism to enhance the image after corrections for 
distance and hardware have been applied. In practice, the boost 
and intensity correction factors and the overall transfer function 
V are implemented using lookup tables. 
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A second embodiment currently in use is creation of a 
displayable reflectance image 270 using the TIFF graphics format. 
Additional industry standard and nonstandard formats can be used 
with no difference in operation to the fundamental concepts 
5 encapsulated in the present invention. 

A third embodiment of the reflectance image 2 70 involves 
merging the reflectance measurements 50 with color intensity data 
280 captured by a traditional or digital photographic camera. The 
resulting image is a colorized reflectance image 290 and is 
displayed in the same industry standard formats as the gray- scale 
images . 

Collectively, the distance measurements 4 0 and the 
reflectance measurements 50 create a point cloud 30. Each point in 
the point cloud 3 0 corresponds exactly to one point in the 
reflectance image 270. 

Other embodiments of a 3D camera can be used with the 
present invention assuming that the other embodiments include an 
ability to generate spatially dense distance data, reflectance 
measurements 50, and a known correspondence between the 2D 
reflectance data and the 3D distance measurement data. 

In the first embodiment of the present invention, the 
magnetic media is comprised of an industry standard hard drive that 
is connected to the 3D camera via a SCSI bus. Alternate bus 
schemes can also be used without any change in the core invention. 
25 These can include Ethernet, parallel, Firewire, or any other bus 
architecture that can move binary data from a computing module to 
a magnetic storage medium. The hard drive stores distance 
measurements 40 and reflectance measurements 50 in a binary data 
file. 
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When an operator requests data in the RDM module, the 
binary data file is copied from the magnetic media to the RAM 
resident on the PC running the RDM module, typically through an 
Ethernet link, but also through a local bus resident on the PC if 
5 the data is stored on a local hard drive as opposed to a networked 
hard drive. 



In a second embodiment of the invention, only the 

reflectance component of the data is loaded into the RDM module. 

When an operator requests a measurement of an object, only the 
10 relevant distance measurement data is loaded from the magnetic 
i media to the PC RAM to perform the calculation or satisfy the 

query. The second embodiment allows for more efficient usage of 

computing resources than the first embodiment. 

Figure 3 is a block diagram of the RDM module in its 
15 broadest aspect. Referring to figure 3, a point cloud 30, 
comprising 3D measurement data 40 and reflectance data 50, is 

1 loaded into a Microsoft Windows compatible software interface from 
a magnetic storage media 60. A Graphical User Interface (GUI) 
comprising a set of windows, buttons, and controls, allows a human 

2 0 operator to interact with the point cloud through the reflectance 

image 270. The reflectance image 270 is loaded into an image 
window 3 00 and an operator generates queries into the data set by 
interacting with a set of icons and windowing commands. Queries 
from the operator can include, but are not limited to: 

25 • Location of a discrete point in absolute coordinates 

(example: the location in feet from the plant origin 
of the north-west corner of a pump housing) 
Distances between two discrete locations in the scene 
(example: the distance between a valve and a flange in 

3 0 inches) 
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Perpendicular and straight line distances between two 
edges 

• Diameters, centerlines, and orientation of pipes 
Perpendicular distances between pipe centerlines 

• Dimensions of structural elements (I, C, T, L, and 
square sections) 

• Nominal or standard sizes of components (example: find 
the nominal production size of a pipe in the scene 
rather than the actual size - a 6 inch pipe actually 
measures 6.625 inches in the OD but would be reported 
to the operator as a 6" pipe as this is the standard 
nomenclature) 

■ Dimensions of flanges, nozzles, and other pipe 
connections 

Perpendicular distance between two planes 
2D area measurements (example: given a tank, compute 
the cross sectional area of the tank) 

3D volume measurements (example: given a tank, compute 
the volume of the tank) 

Measurements of all of the above across multiple 
reflectance images 

Location and orientation of planar surfaces 

Operator queries are processed by referencing the 
distance measurements 40 from the reflectance images 270. Applying 
25 a set of measurement algorithms 90 to the point cloud distance 
measurements 4 0 generates the specific geometric data requested by 
the operator. The result of the query is displayed in an output 
window 310 or is stored in a file on a magnetic storage media 60. 

The current embodiment of the RDM 70 is comprised of 4 
3 0 individual windows: Image Window 3 00, 3D window 310, ScanView 
window 320, and the Output window 330. In addition a series of 
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pull down menus allows the user to load files, exit the program, 
select preferences and options, and select controls for making data 
queries . 

The Image Window 3 00 is a display window for the 
reflectance image 270. The reflectance image 270 is loaded into 
the Image Window 3 00 by the RDM 70 from the magnetic storage media 
60. The reflectance image 270 appears to the operator as a gray- 
scale image that can be panned and zoomed. Controls are provided 
in the RDM 70 so that the operator can adjust the contrast, 
brightness, and gamma correction of the reflectance image 270 to 
enhance the viewing of the image when image quality is not 
sufficient for the operator's needs. The operator performs all 
measurement functions by interacting with the Image Window 300. 

The 3D Window 310 is a display window for the point cloud 
30. The 3D window 310 is not required for the present invention 
however it is a useful display so that an operator can review and 
check the point cloud 30. 

The ScanView Window 32 0 is a display window containing a 
list of the available 3D camera 2 0 positions or scans. The scans 
can be loaded into the RDM 70 by double-clicking on the scan name. 
Right clicking on the scan name allows the operator to set 
parameters concerning the particular scan such as range bounds, 
display color, and display position. Scans can be grouped into a 
hierarchical structure or displayed individually. 

The Output Window 33 0 is a data display window that shows 
the results of operator queries. The Output Window 330 contains 
controls that allow the operator to change the system of 
dimensional units used for display. The current embodiment of the 
invention allows display of results in meters, decimal feet, and 
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feet and inches. Other embodiments could display any other 
measurement units system appropriate for the task. The contents of 
the Output Window 33 0 can be saved to a magnetic storage media 60 
for export to CAD tools 110. 

5 An operator can determine the location of a discrete 

point in three-dimensional space by selecting the Pick Point icon 
on the Measurement Toolbar of the RDM 70. After the Pick Point 
icon is selected by the operator, a location is selected by the 
operator in the Image Window 300 that corresponds to the point of 
yiO interest. The RDM 70 uses the image coordinates selected by the 
O operator as an index into the distance measurements 40. The result 
m of the lookup is then converted to the operator selected units 
4* system and is then written to the Output Window 330 for display to 
HI the operator. 

1.15 Schematically, there is 

Jjj (i, j) -> (r, I, y, n) -> (d, azimuth, zenith, F) -> (x, y, z, I') -> (X; Y, Z, F), 

pixel indices, 

raw range measured by laser to imaged spot, 
raw intensity measured by laser to imaged spot, 
raw encoder value for mirror's yoke position, 
raw encoder value for mirror's nod position, 
Distance from center of scanner to imaged spot, 
Azimuth angle to imaged spot from scanner origin, 
Zenith angle to imaged spot from scanner origin, 
Corrected intensity from equation (A) above, 
Cartesian coordinates in the coordinate system local to 
the scanner, 



* Where 

(i, j) 
r 

20 I 

Y 
n 

d 

Azimuth 
25 Zenith 
I 1 

x,y, z 



-17- 

X,Y,Z Cartesian coordinates in the world coordinate system. 

The progression is from raw values, through calibrated 
values, and finally on to world or user coordinates. 

The first ordered pair, (i, j) is selected as the user 
interacts with the Image Window 300. The second ordered quadruple 
is raw data as sent by the scanner and stored on magnetic media in 
a file. The first two, r and I, are respectively range and 
intensity measured by the laser system. The latter two, y and n, 
are encoder values read directly from the positioning of the 
mirror. 

The next set consists of calibrated data. The intensity 
is calibrated via equation (A) . Since the laser produces 
calibrated data already, all that remains is to subtract the offset 
distance from the laser zero-point to the scanner zero-point at the 
center of the mirror. That is, d = r - r 0 . 

For the angles, a more complicated scheme is employed: 

Azimuth = A,y + Ao + A1 sin(A,y + cc) , 
Elevation = Eo + -2 Xn + B sin(A,n + p) + C sm(ky + 8) . 

In these equations the raw, measured quantities are 

2 0 Xn = nod Angle = (nod encoder count) X (nod resolution), 

Xy = yoke Angle = (yoke encoder count) X (yoke resolution). 
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The set of calibration parameters Ao, A1, a, Eo, B, p, C, and 6 are 
determined according to our calibration process prior to the field 
scanning. 

Once the calibrated polar data has been determined, it is 
simple to construct the Cartesian values via the usual formula: 

x = d cos(azimuth) sin(zenith), 
y = d sin(azimuth) sin(zenith), 
z = d cos (zenith) . 

The world coordinates may be determined from these local 
coordinates, viz. : 

(X, Y, Z)~T = R( (x,y,z)~T) + t, 

Here "~T' denotes transposition and 

R = the matrix that rotates from scanner local into world 
coordinates, 

t = the (world) location of the scanner origin. 

The calibration parameters are stored in a header section 
at the beginning of the raw data file, and the (R, t) 
transformation information is stored within an auxiliary control 
file. 

An operator can measure the relative distance between two 
points in three-dimensional space by selecting the Measure Distance 
icon on the Measurement Toolbar of the RDM 70. After the operator 
selects the Measure Distance icon, a first location is selected in 
the Image Window 300. After the first location is selected, the 
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operator selects a second location by moving the mouse pointer over 
the Image Window to the location of the end point of the 
measurement . 

For each location selected, the three-dimensional location 
5 in space is determined from the correlation between the reflectance 
image 270 and the distance measurements 40. Initially the 
coordinate information for each location is computed relative to 
the local coordinates of the 3D camera 20 but is subsequently 
transformed into world coordinates using the location and 
10 orientation parameters for the corresponding 3D camera 2 0 position. 
Differences are computed for the x- coordinates, y-coordinates, and 
z-coordinates. The absolute distance between the specified points 
is computed using the Pythagorean theorem. Mathematically, we have 

Given 3 -vectors p and q, where 

p = first picked point, in coordinate system local to 3D camera 
location P, 

q = second picked point, relative to coordinate system local to 3D 
camera location Q, 

we compute 

20 w p = R P (p) + t P , and w q = R Q (q) + t Q . 

Then 

W D = p- q. 

The "delta-difference' values are then the ordered triple W D = 
(D x , D y , D z ) with the total, or absolute distance given by 



25 



|D| = V(D X 2 + D y 2 + D z 2 ). 
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The pre-superscript "W denotes quantities in the world 
coordinate frame. The transformation from local scan coordinates 
to world coordinates is accomplished via a rotation R followed by 

a translation t. It is possible to have two different transforms 

5 if the points are selected from different 3D camera locations. 

The results of the measurement calculations are then 
converted to the operator-selected units system and are written to 
the Output Window 33 0 for display to the operator. 

-;; - 

p An operator can measure the relative distance between two 

HJlO edges in three-dimensional space by selecting the Measure Edge icon 
hj on the Measurement Toolbar of the RDM 70. After the operator 
lAl selects the Measure Edge icon, a first edge is selected in the 
Image Window 300. After the first edge is selected, the operator 
Q selects a second edge by moving the mouse pointer over the Image 
SHl5 Window 3 00 to the location of second edge. 

y The selection cursor for measuring between two edges is 

comprised of a box. The operator need only include the edge 
somewhere within the area of the box. A calculation is performed 
to generate the actual location of the edge at accuracy greater 
20 than a single pixel. 

The single-edge extraction algorithm operates on the 
distance measurements 3 0 that correspond to the reflectance image 
270 pixels within the box. For each pixel row in the selection 
box, the discontinuity point is determined by looking for jumps in 
25 the distance measurements 30 greater than a specified, configurable 
threshold (10 cm default.) For each pixel row the last valid pixel 
location before the discontinuity is stored. 
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After all pixel rows within the selection box have been 
similarly processed, the best -fit line passing through the stored 
pixel locations is determined using a linear least-squares 
technique : 

5 First, outlier points are filtered from those pixels 

selected in the Image Window 300. The remaining points are 
averaged. The following matrix is constructed: 

M = Z (p, -C) T (p 7 -C), 

Where 

p,= 3D coordinate of the reflectance image pixel in question, and 
C = Arithmetic average of all remaining pixel locations. 

The sum is performed over all stored pixels. The 
direction of the line representing the edge is chosen as the 
eigenvector corresponding to the minimum eigenvalue of the matrix 
M. Taking the vector c as a point on the line completes the 
specification of the edge in 3 -space. The second edge is 
determined in identical fashion. 

In the following discussion, the first edge selected will 
be referred to as the "reference edge' and the second edge will be 
20 referred to as the "query edge.' The perpendicular distance 
between the two computed edges is measured relative to the three- 
dimensional location of the reference edge. However, since it is 
not generally true that the reference edge and the query edge are 
parallel, the distance is measured from the query edge to a special 
25 point on the reference edge, called the reference point. The 
reference point is that point on the line comprising the reference 
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edge that is closest to the center reflectance image pixel in the 
selection box used to establish the reference edge. 

If the two edges have orientations more that 10° apart, 
the operator is warned and the calculation is discontinued. The 
5 cutoff value of 10° is not fixed and may be changed by the 
operator . 

The center distance between the two edges is computed as 
the distance from the reference point to the query edge. Note that 
in the case whereby the reference edge and the query edge are truly 
10 parallel this distance is equal to the parallel-line distance 
between the two edges . 

The results of the edge measurement calculations are then 
converted to the operator- selected units system and are written to 
the Output Window 330 for display to the operator. 

15 An operator can measure the diameter of a pipe by 

selecting the Measure Pipe icon on the Measurement Toolbar of the 
RDM 70. There are currently two embodiments of pipe diameter 
measurement . 

In the first embodiment of pipe diameter measurement, the 
2 0 Measure Pipe icon requires the operator to select the orientation 
of the pipe (north-south, east-west, vertical) . The operator then 
selects a region of the pipe in the Image Window 300 using a 
dragged selection box. Alternately the operator can select a 
region using a fence method. 

25 The distance measurements 4 0 that correspond to the 

reflectance image 270 pixels within the selection box are projected 
onto the 2D plane perpendicular to the pipe orientation that has 
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been selected by the operator. A non- linear least -squares 
algorithm then solves for the best -fit circle that passes through 
the distance measurements selected. The algorithm uses a 
Levenberg-Marquardt technique with kernel function 

5 E(R, C) = Z (|P r C|-R) 2 . 

Where 

R = radius of the circle, 

C = circle origin in 2D coordinate frame, and 
P ; = the 2D projected coordinates of the 1 th distance 

measurement . 

The sum is taken over all selected distance measurements. 
The initial value for the circle origin is taken as the average 
location of the contents of the selection box, and the initial 
radius is user-selectable but 30 cm works well as the default. 

f||15 After convergence, the circle origin is converted back to 

three-dimensional world coordinates using the inverse of the 
initial transformation. The ambiguity in projecting from the 
lower-dimensional space into the higher-dimensional space means 
that the point is projected into a line. This ambiguity is 
2 0 resolved by choosing, for the circle origin, the point on the line 
closest to the arithmetic mean of the originally selected 
reflectance image pixels. 

The pipe centerline is chosen as the line parallel to the 
operator-selected pipe orientation that passes through the circle 
25 origin. The pipe diameter is calculated as twice the circle 




-24- 



radius R determined above. The results of the calculations are 
then converted to the operator- selected units system and are 
written to the Output Window 33 0 for display to the operator. 

In the second embodiment of the pipe diameter 
5 measurement, the operator selects the Measure Pipe icon on the 
Measurement Toolbar then indicates a single point on the pipe of 
interest in the Image Window 300. In the second embodiment it is 
not necessary for the operator to select the pipe orientation or 
select a portion of the pipe with a selection box. 

10 Initially, a small region surrounding the selected 

reflectance image pixel is examined for consistent distance 
measurements. If the distance measurements are not consistent, the 
operator is notified and the diameter calculation algorithm is 
aborted. 

15 The initial region is chosen as a 5x5 -pixel square 

surrounding the selected reflectance image pixel, though the actual 
size is operator configurable. 

From the initial region an initial estimate of the pipe 
size, location, and orientation is determined using a Levenberg- 
20 Marquart algorithm appropriate for cylinders. The algorithm also 
provides error bounds on these parameters. 

The first iteration of the Levenberg -Marquart algorithm 
examines all distance measurements 4 0 not previously considered 
that are adjacent to distance measurements 4 0 used thus far to 
25 compute the pipe parameter. Distance measurements 40 not within 
the parameter space of the initial pipe estimate as delimited by 
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the error bounds are rejected. Those not rejected are used to 

refine the pipe parameters and errors on those parameters. 

Subsequent iterations continue until no new points are 
found. The operator is informed if errors are not acceptable for 
5 the pipe parameters. 

The results of the pipe diameter and centerline 
measurement calculations are then converted to the operator- 
selected units system and are written to the Output Window 33 0 for 
display to the operator. 

5 

rilO An operator can measure the distance between the centers 

Jp of two pipes by selecting the Measure Pipe Centers icon on the 
W Measurement Toolbar of the RDM 70. The RDM assumes that the 
2 operator has generated the diameter measurement of the two pipes 
^ previously. 

pes 

fUi5 In the following discussion, the first pipe will be 

referred to as the "reference pipe' and the second pipe will be 

Sj referred to as the "query pipe. 1 Perpendicular distances are 
measured relative to the reference pipe. However, since it is not 
generally true that the reference pipe and the query pipe are 
20 parallel, the distance is measured from the query pipe to a special 
point on the reference pipe, called the reference point. The 
re f erence point is that point on the centerline of the reference 
pipe that is closest to the initial reflectance image 270 pixel 
used to establish the reference pipe. If the reference pipe was 
2 5 established by selecting a region, the reference point is that 
point on the centerline of the reference pipe that is closest to 
the arithmetic average of the initial reflectance image 270 pixels 
used to establish the reference pipe. 



-26- 



If the two pipes have orientations more that 10° apart, 
the operator is warned and the calculation is discontinued. The 
cutoff value of 10° is not fixed and may be changed by the 
operator. 

5 The center distance between the two pipes is computed as 

the distance from the reference point to the query pipe 1 s 
centerline. Note that in the case whereby the reference pipe and 
the query pipe are truly parallel this center distance is equal to 
the parallel-line distance between the two pipes. 

□10 The results of the center distance measurement 

H calculations are then converted to the operator- selected units 
£ system and are written to the Output Window 33 0 for display to the 
operator. 

An operator can measure the centerline and dimensions of 
rfl5 standard structural elements. The typical list of structural 
rU elements includes I-beams, T-beams, C-beams, L-beams and 
2 square/rectangular section beams. Important dimensions of these 
fy items include centerlines and flange and web dimensions. The 
operator can generate dimensions of structural elements by 
2 0 selecting the appropriate section shape on the Structural Elements 
tool bar on the Measurement Toolbar of the RDM 70. Once the 
operator selects the section shape icon they then select points on 
the desired element as instructed by a sequence of RDM prompts. 

For example, to measure an I-beam the operator selects 
25 the I -Beam icon. The RDM 70 prompts the operator to select a point 
on the exterior face of the beam near an edge as represented by a 
pixel in the reflectance image 270. If this operation is 
successful, the operator is prompted to pick another point in the 
reflectance image 270 on the same face, but near the opposite edge. 



This second point should be at approximately the same longitudinal 
position as the first point. If the second operation succeeds, the 
operator is prompted for a final time to select a point in the 
reflectance image 2 70 on the opposite side of the I-beam, again 
near an edge. This last pick is in the interior of the beam (see 
diagram.) Other structural elements are done similarly. The table 
below lists the type of structural element with the requisite input 
values and computed output values. 

After the initial identification of key features, the 
subsequent identification and measurement proceeds identically for 
all structural elements. First, straight edges are extracted for 
each picked point using the technique outlined for measuring 
between edges. A check is performed to guarantee that these edges 
are indeed parallel to within a few degrees (an adjustable 
parameter.) If the edges are not essentially parallel, the 
algorithm reports an error and aborts. 

Next, a plane is constructed so as to be perpendicular to 
the weighted average of the directions of the straightedge lines. 
Mathematically the plane can be described using a normal N and 
distance d such that 

N ■ x = D for all points x lying within the plane. 

There is 

N = w x l x + w 2 1 2 + w 3 1 3 / (wl+w2+w3) , and 
D = N • (w 2 p x + w 2 p 2 + w 3 p 3 ) , 



Where the w ± are the weight factors. 
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The points p l7 p 2 and p 3 are the original three pick-points and 
the corresponding straight edges are represented 
parametrically as 

L ± s + q ± , 

With 1^ the (unit) direction vector of the line and q ± a point on 
the line. 

The plane is then intersected in turn with each of the lines to 
determine three "control 1 points Ci from which the measurements 
will be computed. Since the lines are almost parallel and the 
plane is approximately perpendicular to all the lines, this 
intersection is possible and can be determined by solving 

D = N • ( ^ s + q ± ) = s (N • L ± ) + N • q ± . 

for s, and back-substituting to obtain the control point, viz. 

S ± = (D - N - qj / (N • LJ , 

so that 

C ± = L ± S ± + q ± . 

The required dimensions are computed from appropriate pairs using 
the algorithm presented above for measuring distances between 
discrete points. 



Element 


Input 


Output 


I-beam 


Three edges. The first two 1 
must be on the short end of the 
beam. The last edge is on the 


Orientation (vertical, north- 
south, east-west, other) 
Location in world coordinates 





other side. 

A point on the interior web 
may be used to determine web 
thickness. 


jjepin 
Width 


T-beam 


Three edges: On top-of-T end, 
on bottom-ot-l end, ana one at 
the junction point. 


Orientation (vertical, north- 
soutn, east- west, ouierj 
Location in world coordinates 
Top length 
Height 


C-beam 


Two edges: one on each end of 
the C . One point on the 
interior face. 


Orientation (vertical, north- 
south, east-west, other) 
Location in world coordinates 
Depth 
Width 


L-beam 


Three edges: One on each end 
or tne l, ana tne inira ai uie 
junction point. 


Orientation (vertical, north- 

oUULU, CaoL-WCol, UlilClJ 

Location in world coordinates 

Depth 

Width 


Rectangular-section beam 


Three edges, in any order. 


Orientation (vertical, north- 
south, east-west, other) 
Location in world coordinates 
Depth 
Width 



An operator can determine the nominal size of a 
structural component. This action is initiated by the operator 
selecting the Nominal Size preference before developing the 
measurement of a standard structural element as described above. 
Upon completion of the structural measurement calculation, the RDM 
70 will compare the resulting dimensional information to a table 
stored in the RDM 7 0 of standard manufactured shapes and structural 
elements. If a close fit is found on the table, then the 
corresponding standard size is reported to the Output Window 33 0 
along with the other dimensional information from the structural 
element measurement calculations. 
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For each type of element there exists a table of standard 
configurations for intrinsic parameters. The table is organized in 
dictionary order to facilitate searching. Once an element is 
identified and the intrinsic parameters are determined, a 
5 comparison is made with the relevant table. A weighting factor is 
assigned for each parameter. The entry that matches best and has 
the overall discrepancy less than 10% (an adjustable parameter) is 
reported to the operator by the RDM 70. 

An operator can measure the center, surface plane, and 
jJLO orientation of a pipe flange or a nozzle by selecting the Measure 
O Flange icon on the Measurement Toolbar of the RDM 70. 

The operator is prompted to select pixels in the 
reflectance image 270 that correspond to the flat planar surface of 
the flange. At least 3 points are required but the operator may 
pick additional points for improved accuracy. These pixels are 
used to reference the distance measurements 4 0 that are used to 
determine a plane, using the best -fit plane algorithm described 
below. If the fit is inconsistent an error is displayed and the 
calculation is aborted. 

The operator is then prompted to select several pixels in 
the reflectance image 270 running across the face of the flange. 
Two pixel locations on the portion of the flange closer to the 
initially picked plane are selected first, then two pixel locations 
on the other half of the flange are selected. These four locations 
25 are sorted according to distance from the initial plane and 
therefore need not be picked in sequence. However, they must be 
along a line approximately parallel to the flange 1 s centerline. 
The points nearest to the flange's face should be placed 
symmetrically about the face. 
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Using the algorithm outlined above for measuring the 
diameters of pipes, the diameter, location, and orientation of the 
centerline of the flange is determined. A best -fit line is 
constructed that passes through the four points picked across the 
5 flange face. The direction of this line is checked for consistency 
with the normal to the previously- computed surface plane. The two 
locations closest to the face of flange are averaged and projected 
onto the centerline to determine the operating center of the 
flange . 

Denoting the centerline as L c s + q c and the points as a 
the projections are given by 

P a = L c (a - q c ) • L c + q c , and 
P b = L c (b - q c ) • L G + q c . 

fU The results of the flange or nozzle measurement calculations 

Wis are then converted to the operator- selected units system and 
m are written to the Output Window for display to the operator. 

An operator can measure the perpendicular distance 
between two planes by selecting the Measure Planes icon in the 
Measurement Toolbar of the RDM 70. The operator then selects two 
2 0 flat locations in the reflectance image 270 in the Image Window 
300. The RDM 70 then generates a first plane by fitting the best 
plane to an array of distance measurements 4 0 that correspond to 
and surround the pixel selected at the first location in the 
Image Window 3 00. 

25 The distance measurements 40 are averaged. The following matrix 
is constructed: 



U10 



m and b, 



M = 2 (P, -C) T (P; "C), 



Where 

p.= 3D coordinate of the distance measurement in question, and 
C = Arithmetic average of all remaining distance measurements. 

The "T 1 superscript denotes matrix transposition and the 
sum is performed over all selected distance measurements 40. The 
perpendicular to the plane generated is a vector proportional to 
the minimum eigenvalue of the matrix M. The normal is chosen to 
have unit length. The plane is fully specified by choosing c to 
lie within the plane. 

The RDM 70 then generates a second plane by fitting the 
best plane to an array of distance measurements 4 0 that surround 
the reflectance image pixel selected at the second location in the 
Image Window 3 00. 

In the following discussion, the first plane will be 
referred to as the "reference plane and the second will be referred 
to as the "query plane.' Perpendicular distances are measured 
relative to the reference plane. However, since it is not 
generally true that the reference plane and the query plane are 
parallel, the distance is measured from the query plane to a 
special point on the reference plane, called the reference point. 
The reference point is that point on the reference plane that is 
closest to the initial reflectance image 270 pixel used to 
establish the reference plane. If the reference plane was 
established by selecting a region, the reference point is that 
point on the reference plane that is closest to the arithmetic 
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average of the initial reflectance image 270 pixels used to 
establish the reference plane. 

If the two planes have orientations more that 10° apart, 
the operator is warned and the calculation is discontinued. The 
5 cutoff value of 10° is not fixed and may be changed by the 
operator. 

The center distance between the two planes is computed as 
the distance from the reference point to the query plane. Note 
U that in the case whereby the reference plane and the query plane 
OlO are truly parallel this center distance is equal to the parallel - 
line distance between the two planes. 

ft The perpendicular measurement between the two planes is 

5 then converted to the operator- selected units system and is written 
L to the Output Window 330 for display to the operator. The relative 
El5 orientation of the query plane relative to the reference plane is 
pJ also reported to the operator. 

fij An operator can measure areas of two-dimensional surfaces 

by selecting the Measure Area icon in the Measurement Toolbar of 
the RDM 70. In the Image Window 300 the operator places a fence or 
20 box about the reflectance image pixels of interest. The surface 
area is computed approximately using the distance measurements 
corresponding to the selected reflectance image pixels and 
knowledge of the angular resolution of the 3D camera 2 0 as 

s - KZrf cos(6 } ). 

2 5 Where 



-34- 

A = Angular pixel size = (Azimuth resolution in radians) * 
(Elevation resolution in radians) , 
r } = Range to the i th pixel, and 

0, = Elevation to the i th pixel. 

5 The sum is taken over all pixels within the region of interest. 

The area measurement is then converted to the operator- 
selected units system and is written to the Output Window 33 0 for 
display to the operator. 

As a result of the 3D camera 40 positions being 
registered to a common coordinate origin, the measurement tools 
contained within the RDM 70 are also useable across multiple 
images. This feature allows measuring behind and through objects 
and over long distances. To use this capability the operator loads 
more than one reflectance image 270 into the RDM 70 at one time. 
Each additional reflectance image 2 70 is contained in its own Image 
Window 300. Selection operations area made between the multiple 
Image Windows . 

Because of the aforementioned common registration, it is 
also possible to measure volumes within the RDM 70. After 
2 0 selecting the volume measurement tool the operator is prompted to 
select points on the border of the object to be measured. Using 
several different reflectance images 270, the operator picks 
sufficient points to create a three-dimensional outline of the 
volume to be measured. The convex hull of these exterior points is 
25 computed and the volume calculation follows directly. 



Each measurement initiated by the operator in the RDM 70 
is recorded in an internal memory buffer, A Measurement Window 
provides a list of currently active measurements. From this 
Measurement Window the operator can delete unneeded or obsolete 
measurements. In addition, the operator can notate each 
measurement with a text message of up to 256 characters. 

This internal memory buffer may be saved in a variety of 
formats, including text (ASCII) and the industry- standard Drawing 
exchange Format, DXF. This latter format may be converted to other 
CAD representations as well including DWG and DGN. 

The measurements, including annotations, may then be 
imported into a variety of CAD programs to facilitate the 
measurement and design processes. 

Although the invention has been described in detail in 
the foregoing embodiments for the purpose of illustration, it is to 
be understood that such detail is solely for that purpose and that 
variations can be made therein by those skilled in the art without 
departing from the spirit and scope of the invention except as it 
may be described by the following claims. 



